The present invention relates to systems (procedures and/or devices) for autofocusing according to the content of an image and, more particularly, to an autofocusing system for microscopes and similar instruments which is insensitive to noise.
Microscopes and other optical instruments need to focus on an object of interest. Preferably, only one focusing step is performed for each object and, once focused, the object will remain in focus. Ideally, for an instrument which scans an object, the object remains in focus while scanning is performed. However, the object analyzed by optical instruments is typically not flat. In addition, not all objects of interest are situated at the same depth with respect to the objective lens. Accordingly, some refocusing is required for such scanning instruments.
With scanning instruments, once a crude focus position has been found, and if the scanning of the object is done in small steps, fine focus adjustments can be sufficient to keep the object in focus so long as the adjustments are performed after each step (movement). For a microscope""s objective lens with a numerical aperture of 1.3, a fine focusing adjustment with an accuracy of about xc2x10.1 micrometers can be required after each step. The procedures used for crude and fine focusing can be the same, although the range of the search and the search step length are usually different. However, if the fine focusing procedure is computationally too intense, another, less intense, more approximate, procedure may be used for crude focusing.
A typical autofocusing procedure system primarily: (1) estimates what the focus (distance and/or angle of the objective lens relative to the object) should be; and (2) mechanically adjusts the distance and/or angle by the focusing mechanism using a predetermined search strategy which is implemented using a controller. Some autofocusing systems use image content information to estimate the focus. Such systems are often called xe2x80x98passive autofocusing systemsxe2x80x99. In such an image content autofocusing system, the image detected by the microscope is provided to a subsystem which estimates the focus.
One benefit of image content autofocusing systems is that the calculations can be performed in software, that is, extra hardware is not needed, thereby reducing the cost of the system. Also, with an image content autofocusing system, any discrepancy (drift) between the estimated focus calculated by the focusing subsystem and the ability of the focusing mechanism to implement the estimated focus is detectable and can be eliminated. Typical applications for image content autofocusing systems include inspecting chips in the semiconductor industry and all kinds of medical microscopy. In addition, if an image content autofocusing system does not need to have highly repeatable focusing mechanics, the focusing mechanism cost can be considerably reduced.
The performance of an image content autofocusing system is primarily limited by the desired search speed for the instrument and the noise-sensitivity of the focus estimation procedure. The search speed depends, among other parameters, on: (1) the rate at which images of the object are acquired (video frame rate); (2) the speed at which the focusing mechanism can respond (mechanical time constants); (3) the search range, once focused; (4) the search step length; (5) the amount of time required for image processing; and (6) the computing power available. In addition, interference from noise must be taken into account by the focus estimation procedure or the optimal focus can not be achieved. In other words, if the focus estimation subsystem is noise-sensitive, the focus estimate will not correspond to the optimal (maximum) desired focus position.
Two typical image content autofocusing procedures which are noise-sensitive maximize the statistical variance of the content of the raw image or the statistical variance of a high-pass filtered version of the image content. Despite the noise-sensitivity, these statistical variances can be used for finding a crude focus position.
To correct for the noise-sensitivity, some conventional procedures measure the statistical variance at multiple foci in the search range, and then using curve-fitting techniques better estimate the optimal (maximum) focus for the objective lens. See, for example, U.S. Pat. No. 5,790,710.
However, such post-processing (a posteriori) curve-fitting techniques have several drawbacks: (1) the mechanical step lengths of the focusing mechanism must be precise for the curve-fitting technique to be accurate; (2) the optimal focus is not known until after all the images used by the procedure have been taken; and (3) the optimal focus may be near the edge of the curve, which can adversely affect the curve-fitting.
Correcting for these drawbacks is usually costly in terms of time or expense because, for example: (1) a more expensive focusing mechanism must be used for precise positioning; (2) more storage may be required to store all the images until the image corresponding to the optimal focus can be determined; and (3) the entire procedure may have to performed again, starting at a different focus estimate, to ensure that the optimal focus is not near the edge of the curve. Although one solution to having more storage takes advantage of a precise positioning focusing mechanism required for other purposes, going back to grab an image at the optimal focus after the post-processing can have problems caused by hysteresis in the control of the focusing mechanism and can take longer to locate the optimal focus.
This invention provides solutions to the problems encountered with conventional autofocusing systems by employing a noise-insensitive filter which operates on image signal data from xe2x80x9cgrabbedxe2x80x9d images of the already existing microscope camera.
One goal of the invention is to provide an autofocusing system which uses a low-cost focusing mechanism by having a filter calculator employ a filter capable of achieving the optimal focus using the content of the image (focus function) without the need for any curve-fitting technique to maximize the measured amount of focus (focus metric) from an energy calculator.
Another goal of the invention is to minimize the amount of time required to obtain a sharp focus.
One object of the invention is to provide a device for determining a focus between an image sensor and an object having: (1) an image sensor for receiving an image of the object and for generating an image signal from the image, the image signal having an image component and a noise component; (2) a filter calculator for receiving the image signal and for generating a filtered image signal such that the noise component of the image signal has been reduced, the noise component being reduced by increasing the energy contributions of parts of the image signal which contribute a relatively larger proportion to the image component than the noise component and by decreasing the energy contributions of other parts of the image signal which contribute a relatively larger portion to the noise component than to the image component; (3) an energy calculator for receiving the filtered image signal and determining an energy level of the filtered image signal; (4) a controller for receiving the energy level and for generating a position control signal in accordance with the energy level; and (5) a positioning mechanism for receiving the control signal and performing a focus adjustment based on the control signal. This filter calculator can also have a linear convoluter; and a filter selected from the group consisting of: a Wiener filter, a modified second order difference filter, and a bandpass filter. The filter can be a one-dimensional array. Also, the filter calculator can be limited to only operate on: (1) the image signal in accordance with a single direction in the image; and/or (2) portions of the image signal corresponding to selected sections of the image.
In addition, the device can include a filter generator for determining the filter characteristics. The filter generator can have: (1) a frequency spectrum generator for receiving image signals and for generating image frequency signals; and (2) a filter optimizer for selecting those characteristics which maximize the signal-to-noise ratio of at least a portion of the image signal. The filter generator can also be a modified second order difference filter generator for selecting the modified second order difference filter from a group of modified second order difference filters which yields the maximum ratio of peak energy level to peak energy width for a plurality of images. Furthermore, the modified second order difference filter can be a first negative value, one or more first zero values, a positive value, one or more second zero values and a second negative value. Also, the number of first and second zero values can be limited to 15.
Another object of the invention is to provide a method for determining a focus between an image sensor and an object having the steps of: (1) generating an image signal from an image of the object, the image signal having an image component and a noise component; (2) filtering the image signal to generate a filtered image signal such that the noise component of the image signal has been reduced by increasing the energy contributions from parts of the image signal which contribute a relatively larger proportion to the image component than the noise component and by decreasing the energy contributions from other parts of the image signal which contribute a relatively larger portion to the noise component than to the image component; (3)generating an energy level from the filtered image signal; (4) generating a position control signal from the energy level; and (5) changing the focus according to the position control signal. Further, selected parts of the image signal where the energy contribution is increased, can be the lower frequencies. The method can also include the steps of convolving a filter with the image signal, the filter selected from the group consisting of: a Wiener filter, a modified second order difference filter, and a bandpass filter. In addition, the method can include the steps of: (1) generating a filter from frequency spectra of one or more image signals; and (2) optimizing the filter by selecting those characteristics which maximize the signal-to-noise ratio of at least a portion of the image signals. Also, the method can include the step of generating a modified second order difference filter by selecting the modified second order difference filter from a group of modified second order difference filters which yields the maximum ratio of peak energy level to peak energy width for a plurality of image signals. In addition, the filtering step can include the steps of: operating on the image signal only in accordance with a single direction in the image; and/or operating only on selected sections of the image signal. Further, the method can include the step of generating a modified second order difference filter by selecting the number of zeros to place between each endpoint and the center of the modified second order difference filter.
These objects and other objects, advantages, and features of the invention will become apparent to those skilled in the art upon consideration of the following description of the invention.